Event

A test level deviates from the planning

Possible causes

The causes may lie in the process before the test process. For instance lower quality of test basis/object, late delivery by design/building, test infrastructure not installed in time, installation of test object in test environment delayed (significantly). The cause may also lie in the test level itself. Think of inadequate resources, incorrect planning, or inadequate progress monitoring.

Consequences for the test process

Successive test levels may start later.

Possible countermeasures

·         In iterative development in consultation with the developers: Limiting system functionality

·         Deploying extra test capacity

·         Successive test levels start in parallel with the ongoing test level, resulting in some inefficiency (two test levels detect the same defect)

·         It is decided to merge certain test levels or types, executing them together

·         Requesting additional budget

·         Reassessing overall test strategy, taking extra risk

·         Reassessing the project planning

·         …

Event

A test level has no further time/resources to execute all required (re)tests

Possible causes

Here, too, the causes may lie outside the testing, cf. above. But internal causes are possible, too: inadequate planning and estimate, unexpected shortage of resources, insufficient productivity.

Consequences for the test process

·         Successive test levels receive a test object of inadequate or uncertain quality

·         Successive test levels are delayed or must stop after inadequate testing

·         Backlog effect: the project seems to be continuing at full speed in a way that conforms to the plan, but an increasing backlog of incomplete work/inadequate quality emerges. Only in that case will the team stop to assess the situation and recognise the enormous quantity of rework and delays.

Possible countermeasures

·         Deploying extra test capacity

·         Allowing a delay of the relevant test level, cf. above

·         Requesting additional budget

·         Reassessing overall test strategy, taking extra risk

·         Reassessing the project planning

·         …

Event

Uncertainty about the scope of the test levels

Possible causes

·         Lack of information when creating the overall test strategy

·         Inadequate alignment when creating the overall test strategy

·         Progressive insight or changed functionality/scope of project makes new testable aspects clear

Consequences for the test process

·         Certain aspects are not tested, or

·         Extra time/money/resources needed to test the aspect

·         Delays in planning

Possible countermeasures

·         Realigning the scope of test levels, reassessing the overall test strategy

·         Deploying extra test capacity for the relevant test levels

·         Allowing a delay of the relevant test level, cf. above

·         Requesting additional budget

·         Reassessing the project planning

·         …

Event

Relatively large numbers of defects for certain characteristics or object parts in specific test levels

Possible causes

Depending on the cause: inadequate quality of test object or test basis

Consequences for the test process

·         Extra time/money/resources needed for retesting and additional tests

·         Delays

Possible countermeasures

·         Realignment, reassessing the overall test strategy

·         Recommending that prior test and evaluation activities be executed more thoroughly for the relevant characteristics/object parts

·         Deploying extra test capacity for the relevant test level

·         Allowing a delay of the relevant test level, cf. above

·         Planning additional tests in the following test level

·         Requesting additional budget

·         Reassessing the project planning

·         …